Optical transmission systems constitute the basic carrier for most telecommunication systems. Many optical transmission technologies are based on the principle of wavelength division multiplexing (WDM), where the transmission channels are carried by optical signals over different wavelengths or different bands of wavelengths. The optical signals are initially modulated at the transmitter end of the optical network, propagated through the network via an optical link, and then detected at the receiver end. The speed and quality of the optical transmission systems, including WDM systems, play a significant role in defining the overall speed and quality of the telecommunication networks. Therefore, enhancing the performance of optical transmission systems has been a factor in shaping and improving the world's telecommunication services.
The transmission system performance may be characterized by its spectral efficiency and its receiver sensitivity. Spectral efficiency may be defined as the system channel modulation rate divided by the wavelength spacing. For example, a system with a 10 gigabit per second (Gbit/s) modulation rate and 50 gigahertz (GHz) channel spacing has a spectral efficiency of 0.2 (20 percent). A higher spectral efficiency reflects more transmission signal capacity within a fixed bandwidth. In contrast, the receiver sensitivity is a measure of signal tolerance to noise, and usually refers to the lowest signal power wherein a certain bit error rate (BER), for example 10−9, can still be detected. Traditionally, modulation formats, such as return-to-zero (RZ) and non-return-to-zero (NRZ), have been applied in optical transmission systems. More recently, optical transmission systems have used an optical duobinary (ODB) modulation format, which has a narrower transmission spectrum and thus higher spectral efficiency and larger chromatic dispersion tolerance. These features have made ODB modulation more preferable than traditional modulations such as NRZ for 10 Gbit/s and higher bit rates transmission systems. Unfortunately, conventional ODB modulation formats suffer from low receiver sensitivity and hence high BERs.